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There  have  been  a few  studies  that  have  dealt  with  procedures  for  the 
development  of  some  generalized  approach,  perhaps  the  most  expansive  studies 
having  involved  the  use  of'the  Position  Analysis  Questionnaire;,(PAQ),  The 
,PAQ;is  a structured  job  analysis  questionnaire  that  provides  for -tl^  ana  lysis 
of  individual  jobs  in  terms  of  each  of  187  job  elements.  On  the  basis  of  a 
series  of  principal , ajmponents  analyses  of  PAQ.  data  a number  of  *job 
dimensions*  Have  beeK;  identified.  Scores  for  jobs  on  these  job  dimensions 
have  been  used  as  the  basis  for^the  prediction  of' test- related  criterion 
values  of  incumbents  on  jobs,  these  studies  employing  data  from  the  nine/. tests 
of  the  General  Aptitude  Test  Battery  (GATB)  of  the  United  States.^ Empl oyment 
Service.  In  certain  other  studies, the  ‘attribute  profiles"  of  the  job 
elements  of  the  PAQihave  been  used  in  conjunction  with  the  PAQ  analyses  of 
jobs"  as  the>predictors  of  the  test-related  criterion  values.-  (The  attribute 
profiles  consist  of  the  median  ratings  on  71  attributes  as  i^ated  in  terms  of 
their  relevance  to  the  individual  job  elements  of  the  PAQJ 

-^The  use  of  PAQ  job  dimension  scores  and  the  attribute  profile  data 
have  indicated  substantial  potential  for  the  use  of  a structured  job  analysis 
procedure  (such  as  the  PAQ)' for  deriving  reasonably  valid  estimates  of  apti- 
tude requirements  of  jobs.  This  generalized  approach  has  been  referred  to  as 
job  component  validity.-. 

The  current  research  program  is  directed  toward  the  further  testing  of 
the  use  of  the  PAQ  for  the  purpose  of  establishuig  the  job  component  validity 
of  tests,  except  that  it  was  directed  toward^^the  prediction  of  test-related 
criterion  values  based  on  commercially-available  tests,  as  contrasted  with 
those  studies  based  on  the  GATB  tests. (which  are  not  available  for  use  by 
private  organizations). 

As  preliminaries  to  the  general  analyses  involved  in  this  project,  a 
special  analysis  was  carried  out  with  the  attribute  profile  data  as  the 
possible  basis  for  the  prediction  of  aptitude  requirements  of  jobs,  involving 
various  methods  for  the  statistical  utilization  of  such  data.  In  addition,  a 
cluster  analysis  was  carried  out  using  a hierarchical  grouping  technique  as 
applied  to  scores  on  13  "overall"  dimensions  of  the  PAQ. 

The  final  analyses  consisted  of  using  as  a sample  jobs  for  which  test 
data  and  PAQ  analyses  were  available.  A separate  analysis  was  carried  out  for 
each  of  five  of  the  "constructs"  represented  by  the  GATB  tests,  the  jobs  in- 
cluded in  each  analysis  being  those  for  which  test  data  were  available  for 
incumbents,  and  for  which  PAQ  analyses  were  available.  In  these  analyses,  a 
comparison  was  made  of  the  predictibility  of  the  test-related  criterion  values 
for  incumbents  as  based  on  the  use  of  job  dimension  scores  for  individual  PAQ 
analyses,  as  contrasted  with  the  predictibility  of  the  test-related  criterion 
values  for  jobs  which  had  been  grouped  into  job  families  using  the  mean  job 
dimension  scores  for  all  of  the  jobs  in  each  of  the  job  families.  Individual 
jobs  were  then  allocated  to  the  job  families  with  which  they  were  most  nearly 
matched  (using  a index).  The  predicted  criterion  values  for  the  job 
families  were  then  "applied"  to  the  individual  job  "assigned"  to  them,  and 
were  then  used  in  the  prediction  of  the  actual  test-related  criterion  values. 

The  results  of  the  analyses  generally  supported  the  potential  use  of 
a structured  job  analysis  procedure  such  as  represented  by  the  PAQ  as  the 
basis  for  the  establishment  of  aptitude  requirements  for  jobs  for  at  least 
certain  aptitudes,  thus  generally  lending  substantial  support  to  the 
practical  use  of  such  a procedure  for  establishing  the  job  component  validity 
of  jobs. 
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INTRODUCTION 


Over  the  years  various  arguments  have  been  set  forth  for  the 
development  and  use  of  some  generalized  approach  to  the  establishment 
of  personnel  requirements  for  jobs.  These  suggestions  have  been  re- 
ferred to  as  generalized  test  validity,  or  synthetic  test  validity. 

The  primary  arguments  for  such  an  approach  have  fallen  into  two 
general  groups.  In  the  first  place,  on  rational  grounds  it  would  seem 
that  those  jobs  which  have  certain  human  behaviors  in  common  should 
also  require  the  same  kinds  of  human  attributes  in  so  far  as  those 
common  behaviors  are  concerned.  The  second  type  of  argument  has 
generally  been  centered  around  practical  considerations.  Often  it  is 
impossible  to  validate  tests  in  each  and  every  job  situation,  and  in 
any  event  the  time  and  cost  of  doing  so  are  prohibitive. 

The  basic  approach  that  would  seem  to  be  common  to  any  such  effort 
logically  would  be  predicated  upon  the  following;  (1)  for  various 
jobs  some  method  of  identifying  the  constituent  components  of  these  jobs 
which  possibly  have  behavior  requirements  in  common;  (2)  a method  of 
determining,  for  an  experimental  sample  of  jobs,  the  human  attribute(s) 
required  for  successful  performance  as  related  to  each  of  those  job 
components;  and  (3)  some  method  of  combining  the  estimates  of  human 
attributes  required  for  individual  job  components  into  an  overall 
estimate  of  human  attributes  requirements  for  an  entire  job.  Such  a 
procedure  would  make  it  possible  to  "build-up"  the  attribute  require- 
ments for  any  given  job  by:  (1)  knowing  what  job  components  occur  in 

the  job  in  question;  (2)  knowing  the  attribute(s)  required  for  each 
such  component;  and  (3)  having  a procedure  for  measuring  the  attributes 
that  are  relevant  to  the  individual  job  components.  Because  of  the 
dependence  of  such  procedures  on  the  identification  of  various  types 
of  relevant  job  characteristics,  it  would  seem  that  the  term  "job 
componenent  validity"  could  well  apply  to  such  a procedure. 

There  have  been  a few  individual  studies  directed  toward  the 
establishment  of  the  aptitude  requirements  of  jobs  on  the  basis  of 
some  such  generalized  approach.  Most  of  these  studies  have  dealt  with 
jobs  within  a certain  restricted  area.  Perhaps  the  most  generalized 
approach  to  this  has  involved  the  use  of  the  Position  Analysis 
Questionnaire  (PAQ)  (McCormick,  Jeanneret,  and  Mecham,  1972). 

The  Position  Analysis  Questionnaire  (PAQ) 

The  Position  Analysis  Questionnaire  is  a structured  job  analysis 
questionnaire  that  provides  for  tno  analysis  of  a variety  of  jobs  in 
terms  of  each  of  187  job  elements.  The  job  elements  are  classified  in 
the  following  six  divisions.  In  each  instance,  an  example  of  a job 
element  is  included. 
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PAQ  Division 

1.  Information  Input 

2.  Mental  Processes 

3.  Work  Output 

4.  Relationships  With  Other  Persons 

5.  Job  Context 

6.  Other  Job  Characteristics 


Illustrative  Job  Element 

Use  of  Written  Materials 

Coding/Decoding 

Use  of  Keyboard  Devices 

Interviewing 

Noise  Intensity 

Responsibility  of  Safety  for  Others 


In  the  analysis  of  jobs  with  the  PAQ,  various  rating  scales  are  used 
with  the  different  job  elements  such  as:  Importance  to  the  Job;  Amount  of 
Time;  Possibility  of  Occurrence  (as  in  the  case  of  accidents);  Extent  of 
Use;  Applicability  (whether  the  job  element  does  or  does  not  apply);  and 
Special  rating  scales. 

Job  Dimensions  Based  on  the  PAQ 

Various  principal  components  analyses  have  been  carried  out  with  PAQ- 
based  data  in  order  to  identify  the  principal  components  that  characterize 
the  structure  of  jobs  (McCormick,  Jeanneret  and  Mecham,  1969;  Marquardt  and 
McCormick,  June  1974.  The  most  recent  of  these  is  based  on  a reasonably 
representative  sample  of  2200  jobs  (Mecham,  February  1977).  His  analyses 
included  separate  principal  components  analyses  of  the  job  elements  within 
each  of  the  six  divisions,  and  an  "overall"  analysis  based  on  all  the  job 
elements  (with  a few  exceptions).  These  analyses  resulted  in  32 
"division"  dimensions  and  13  "overall"  job  dimensions. 

PREVIOUS  RESEARCH  WITH  THE  PAQ 

The  primary  previous  research  with  the  PAQ  in  the  job  component 
validity  frame  of  reference  was  carried  out  with  jobs  for  which  test  data 
for  job  incumbents  were  available  through  the  United  States  Employment 
Service.  (Mecham  and  McCormick,  1969;  and  Marquardt  and  McCormick,  July 
1974).  In  these  studies  two  criteria  were  used  as  indexes  of  the  "impor- 
tance" of  the  attributes  measured  by  the  General  Aptitude  Test  Battery 
(GATB)  to  the  individual  jobs  in  a sample  of  jobs.  One  of  these  consisted 
of  the  mean  test  scores  of  incumbents  on  the  various  job,  and  the  other 
consisted  of  the  validity  coefficients.  In  these  studies  separate  analyses 
were  carried  out  for  each  of  the  nine  tests  of  the  GATB.  In  the  first  of 
these  studies  PAQ  analyses  for  179  positions  were  "matched"  with  90  jobs 
for  which  the  USES  had  published  test  data  for  the  job  incumbents.  (There 
were  multiple  analyses  for  certain  jobs.)  In  the  second  study  PAQ  analyses 
for  a total  of  659  positions  were  matched  with  149  jobs  for  which  the  USES 
had  published  test  data.  In  the  case  of  both  of  these  studies  the  predic- 
tion of  the  mean  test  scores  of  the  incumbents  from  PAQ  job  dimension 
scores  was  quite  respectable.  However,  the  prediction  of  the  validity 
coefficient  criterion  was  not  as  good,  perhaps  at  least  partially  because 
of  the  well -recognized  problems  associated  with  validity  coefficients,  such 
as  poor  criteria,  restricted  range,  etc. 
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In  the  third  analysis  of  this  type  carried  out  by  Mecham  (April  1977), 
data  relating  to  the  PAQ  analyses  were  matched  with  163  jobs  for  which  the 
USES  had  published  test  data.  In  this  study,  however,  instead  of  matching 
individual  PAQ  analyses  with  these  jobs,  all  of  the  PAQ  analyses  that  had 
the  same  9-digit  code  number  from  the  Dictionary  of  Occupational  Titles 
(DOT)  were  "averaged"  to  represent  a "single"  PAQ  analyses  to  be  matched 
with  each  of  the  163  jobs  for  which  the  GATB  test  data  were  available. 

This  procedure  was  used  since  it  was  felt  that  the  "average"  PAQ  job 
dimension  scores  for  various  jobs  with  the  same  DOT  code  number  would 
represent  more  stable  values  for  the  jobs  than  PAQ  analyses  of  individual 
posi tions . 

In  the  studies  by  Mecham  (April  1977)  and  Marquardt  (July  1974)  a 
third  criterion  of  the  "importance"  of  various  tests  to  the  jobs  in 
question  was  used.  This  criterion  was  the  value  one  standard  deviation 
below  the  mean  test  scores  of  the  incumbents  on  the  individual  jobs. 

This  criterion  is  called  "1  SD  below  the  mean,"  or  "mean-SD."  Such 
a value  might  be  viewed  as  a possible  cutoff  score.  Although  test 
cutoff  scores  used  in  personnel  selection  obviously  vary  with  labor 
market  conditions,  it  is  probable  that,  in  general  terms,  scores  one 
standard  deviation  below  the  means  would  more  nearly  approximate 
typical  cutoff  scores  than  mean  scores  as  such. 

The  ranges  and  medians  of  the  multiple  correlations  across  the 
nine  GATB  tests  resulting  from  these  three  studies  are  given  below: 


Mecham  Marquardt 


Cri terion 

and  McCormick 

and 

McCormick 

Mecham 

Mean  test  scores 

Range 

.59  to  .80 

.46 

to 

.76 

.30  to  .83 

Median 

.71 

,73 

.73 

1 SD  below  the  mean 

Range 

.42 

to 

.77 

.24  to  .84 

Median 

.73 

.70 

Validity  coefficients 

Range 

.40  to -59 

.26 

to  .44 

-.02  to  .39 

Median 

.47 

.39 

.13 

Another  approach  to  the  use  of  PAQ-based  data  as  the  basis  for 
estimation  of  aptitude  requirements  for  jobs  involves  the  use  of  what  are 
referred  to  as  "attribute  profiles"  of  the  job  elements  of  the  PAQ.  These 
attribute  profiles  consist  of  the  median  ratings  of  the  "relevance"  of 
each  of  71  human  attributes  to  each  of  the  job  elements.  The  ratings 
were  carried  out  by  industrial  psychologists,  there  being  at  least  eight 
ratings  for  each  of  the  attributes.  (Mecham  and  McCormick,  1969; 

Marquardt  and  McCormick,  1972).  In  a subsequent  study  (Shaw  and 
McCormick,  1976)  several  methods  of  combining  the  attribute  profile  data 
and  the  job  analysis  data  for  individual  jobs  were  used  experimentaly 
as  the  basis  for  prediction  of  test-related  criteria.  In  these  various 
studies,  the  use  of  the  attribute  profile  data  was  reasonably  effective 
in  the  prediction  of  mean  test-score  criteria  of  job  incumbents  on  the 


cognitive  tests,  was  moderately  predictive  of  the  test  data  for  the 
perceptual  tests,  but  was  not  effective  with  the  psychmotor  tests 
(such  as  motor  coordination,  finger  dexterity,  and  manual  dexterity). 

OBJECTIVES  OF  THE  PRESENT  RESEARCH  PROJECT 

As  indicated  above,  the  previous  research  with  the  PAQ  in  the  job 
component  validity  framework  had  involved  the  use  of  PAQ-based  data  for 
the  prediction  of  test-related  criterion  data  for  incumbents  on  the  nine 
GATB  tests.  Since  these  tests  are  not  available  for  use  by  private 
organizations,  it  was  considered  desirable  to  experiment  with  the  use 
of  the  PAQ  as  the  basis  for  the  establishment  of  job  requirements  ex- 
pressed in  terms  of  commerci ally-avail  able  tests.  The  basic  approach 
used  in  the  present  project  was  substantially  the  same  as  that  used  in 
previous  research  in  which  GATB  test  data  were  used,  except  that  in  the 
present  instance  test  data  for  job  incumbents  based  on  commercial ly- 
available  tests  were  used. 


METHOD 

The  object'  s of  the  study  required  the  accumulation  of  test  data 
for  incumben  /arious  jobs,  along  with  PAQ  analyses  for  each  such 
job.  In  th  d,  efforts  were  made  to  obtain  test  validity  and/or 

normative  various  kinds  of  organizations,  for  virtually- any 

type  of  j involving  virtually  any  commercial ly-avail able  aptitude 

test,  or  uesLb  that  resembled  commercially-available  tests.  Various 
types  of  appeals  were  made  to  many  different  organizations.  (These 
approaches  are  discussed  further  in  McCormick,  DeNisi,  and  Shaw, 

May  1977).  It  must  be  stated  that  the  results  of  these  several  appeals 
were  very  discouraging,  resulting  in  the  accumulation  of  appropriate 
test-related  data  for  incuments  on  only  202  jobs.  In  certain  instances 
these  test  data  were  obtained  from  published  sources. 

In  the  case  of  some  jobs  for  which  test  data  were  available,  it  was 
not  possible  to  obtain  PAQ  analyses  of  the  jobs  in  question,  and  in  some 
of  these  instances  PAQ  analyses  for  corresponding  jobs  were  obtained  from 
the  PAQ  data  bank  (which  at  the  time  included  analyses  of  about  25,000 
positions,  representing  1900  different  job  classifications). 

Constructs  Used  In  The  Study 

The  basic  approach  of  the  project  involved  the  development  and  use 
of  regression  equations  consisting  of  PAQ  job  dimension  scores  as 
predictors  of  test-related  criteria  based  on  the  GATB  tests.  Therefore, 
in  considering'  the  test  data  that  had  been  obtained  for  incuments  on 
various  jobs,  it  was  the  intent  to  select  test  data  that  were  based  on 
tests  that  measured  the  same  "constructs"  as  those  measured  by  the  nine 
GATB  tests.  In  this  way  it  presumably  would  be  possible  to  use  the  same 
regression  equations  derived  for  the  GATB  tests  in  the  prediction  of  test- 
related  criteria  for  corresponding  commercially-available  tests.  The 
"matching"  of  commercially-available  tests  with  GATB  tests  was  based  largely 
on  subjective  judgments  of  similarity  of  content  of  the  tests.  Only  in 
certain  instances  were  data  available  on  the  correlations  between 
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the  connercidlly-available  tests  and  the  GATB  tests. 

Conversion  of  Norms 

Since  data  for  one  or  more  commercialiy-available  tests  were  to 
be  used  as  measures  of  each  of  the  "constructs"  represented  by  the 
GATB  tests,  it  was  necessary  to  convert  scores  of  the  individual  tests 
to  a common  metric.  For  this  purpose  a standard  score  system  was  used 
that  consisted  of  a mean  of  100  and  a standard  deviation  of  20.  (This 
is  the  same  standard  score  system  as  used  with  the  GATB  tests.)  The 
GATB  tests  norms  are  based  on  a "general  working  population."  In  the 
case  of  the  commercial ly-availabl e tests  there  were  very  few  norms  based 
on  such  populations,  and  therefore  it  was  necessary  to  "build  up" 
such  a general  norm  for  each  test  from  combinations  of  norms  for  various 
subgroups.  This  method  of  forming  a "general  working  population"  norm 
for  any  given  test  undoubtedly  introduced  some  error  into  the  common 
normative  metric.  Unfortunately  there  was  no  other  acceptable  al- 
ternative available.  This  conversion  was  necessary  for  two  of  the  four 
criteria  of  the  "importance"  of  individual  constructs  to  the  jobs  in 
question. 

Actual  Criteria  Used 


Four  criteria  were  used  in  the  study,  these  different  criteria 
representing  various  indices  of  the  "importance"  of  each  of  the  con- 
structs represented  by  the  GATB  tests  to  the  jobs  in  the  sample.  These 
criteria  for  each  Job  and  test  consisted  of;  (1)  the  mean  test  score 
of  job  incumbents  on  the  individual  job;  (2)  the  test  score  one 
standard  deviation  below  the  mean  of  the  scores  of  incumbents  on  each 
job,  referred  to  as  "1  SD  below  the  mean"  or  "mean-SD;"  (3)  a validity 
coefficient;  and  (4)  an  indication  of  whether  the  test  would  be  "valid" 
for  the  job.  (A  test  was  considered  to  be  "valid"  if  the  data  obtained 
on  that  test  included  a statistically  significant  validity  coefficient. 
If  a validity  coefficient  was  reported  for  a job  but  was  not  statis- 
tically significant,  then  the  test  was  considered  to  be  "nonvalid." 

If  no  validity  coefficient  was  reported  this  criterion  was  of  course 
considered  as  "missing"  for  the  particular  job  in  question).  The 
primary  criteria  of  the  project  were  considered  to  be  the  mean  test 
scores  and  the  scores  one  standard  deviation  below  the  mean  (1  SD 
below  the  mean) . 

Predicted  Criterion  Values 


The  predicted  criterion  values  for  the  individual  jobs  were  ob- 
tained from  standard  computer  printouts  of  data  that  are  generated 
from  the  PAQ  analyses  of  jobs.  The  first  three  predicted  criteria  are 
based  on  the  regression  equations  derived  from  the  analysis  of  the  PAQ 
job  dimension  scores  as  predictors  of  those  criteria  as  based  on  the 
GATB  tests.  The  fourth  criterion  (an  indication  of  whether  the  test 
would  or  would  not  be  "valid"  for  the  job)  re'^lects  essentially  a 
"policy  capturing"  procedure  that  pajaliels  the  practice  of  the  USES 
in  its  approach  to  the  identification  of  the  three  "best"  or  most 
"valid"  tests  for  use  in  the  selection  of  individuals  for  any  given 
job.  A test  was  predicted  to  be  "valid"  if.it  were  one  of  the  three 
tests  identified  as  being  "best"  in  terms  of  the  USES  procedures.  A 
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test  would  be  predicted  to  be  "nonvalid"  if  it  were  not  one  of  these 
three  "best"  tests. 


Cluster  Analysis  of  Jobs 


It  was  planned  to  carry  out  the  analyses  of  the  use  of  PAQ-based 
data  for  the  estimation  of  aptitude  requirements  of  jobs  on  the  basis  of 
PAQ  analyses  of  individual  jobs,  and  also  on  the. basis  of  the  place- 
ment of  individual  jobs  into  job  families  or  clusters. 


Toward  this  end  it  was  then  necessary  to  have  a set  of  job  clusters 
(or  job  families)  that  could  be  used  in  this  pnase  of  the  analysis.  In 
an  earlier  study  DeNisi  and  McCormick  (1974)  had  carried  out  two  cluster 
analyses  of  jobs  as  based  on  PAQ  data.  Although  those  cluster  analyses 
had  been  carried  out  with  PAQ-based  data,  they  had  involved  the  use  of 
an  earlier  set  of  job  dimensions.  Since  the  current  study  involved  the 
set  of  jobs  dimensions  developed  by  Mecham  (February  1977)  it  was  con- 
sidered desirable  to  use  these  as  the  basis  for  a cluster  analysis.  In 
this  regard,  Mecham's  13  "overall"  dimensions  were  used.  The  sample  of 
jobs  consisted  of  746  jobs  that  was  a sub-sample  of  2200  jobs  which  had 
earlier  been  used  in  the  principal  components  analysis  of  PAQ  data  for 
the  derivation  of  the  job  dimensions.  The  job  dimension  scores  on  these 
13  dimensions  for  the  746  jobs  were  subjected  to  a hierarchical  grouping 
procedure  developed  by  Ward  (1961)  and  Ward  and  Hook  (1963). 

In  the  formation  of  job  families  (i.e.,  clusters)  a major  problem  i 

lies  in  making  a decision  regarding  the  number  of  families  to  recognize.  j 

The  more  clusters,  the  more  homogeneous  the  jobs  within  the  clusters.  But  * 

greater  homogeneity  must  be  made  at  the  possible  sacrifice  of  practical  j 

considerations,  since,  for  practical  considerations,  fewer  clusters 

usually  would  be  desirable.  For  purposes  of  this  study  a decision  was  < 

made  to  use  three  sets  of  clusters,  each  set  consisting  of  different  J 

numtjers  of  clusters,  in  order  to  be  able  to  compare  the  predictability  i 

of  the  criterion  values  when  the  predictors  were  based  on  various  numbers  j 

of  clusters  to  which  jobs  were  assigned.  Toward  this  end,  the  three  sets  j 

of  clusters  chosen  were  those  emerging  from  the  iterations  that  resulted 
in  60,  40,  and  20  clusters.  These  clusters  are  illustrated  in  the  report 

by  Shaw,  DeNisi  and  McCormick  (April  1977).  . 


General  Plan  of  Analyses 


In  very  general  terms  the  analyses  that  were  carried  out  can  be 
characterized  in  terms  of  the  following  combination  of  variables:  the 
use  of  PAQ  analyses  of  individual  jobs  vs.  PAQ  analyses  of  job  families 
as  the  basis  for  prediction;  the  types  of  predictors  that  were  used;  and 
the  criteria  that  were  used.  The  various  combinations  of  these  variables 
are  shown  below: 


'I 


Gasis  of  Prediction 


Individual 

jobs 

Job  families 

Criterion 

Job  dimension 

Attribute 

Job  dimension 

Attribute 

scores 

data 

scores 

data 

Mean 

X 

X 

X 

. 

"1  SD  below  mean" 

X 

X 

X 

Not 

relevant 

Validity 

X 

— 

X 

Val id-nonval id 

X 

— 

— 

An  "X"  indicates  those  specific  analyses  that  were  carried  out. 


This  plan  was  repeated  for  the  constructs  represented  by  each  of  five 
of  the  GATB  tests,  namely  :G  (General  Intelligence);  V (Verbal  Aptit'<'e); 

N (Numerical  Aptitude);  S (Spatial  Aptitude);  and  Q(Clerical  Percpetion). 

There  were  insufficient  jobs  for  which  relevant  data  were  available  to 
carry  out  analyses  for  the  other  four  constructs,  namely:  P (Form  Percep- 
tion); K (Motor  Coordination);  F (Finger  Dexterity);  and  M (Manual  Dexterity). 

Two  types  of  PAQ  predictors  were  used,  namely  those  which  consisted  of 
job  dimension  scores,  and  those  which  were  derived  from  the  "attribute 
profiles"  of  the  PAQ  job  elements.  These  two  types  were  used  with  the 
"individual"  PAQ  analyses,  but  only  those  based  on  job  dimension  scores 
were  used  with  the  "job  family"  PAQ  analyses. 

In  connection  with  the  job  families,  the  average  scores  of  all  of  the 
jobs  within  each  family  were  derived  for  the  various  job  dimensions.  These 
"average"  scores  were  then  used  as  the  basis  for  predicting  criterion  values, 
just  as  in  the  case  of  PAQ  analyses  of  individual  jobs.  In  turn,  these 
predicted  criterion  values  were  applied  to  all  of  the  individual  jobs  that 
fell  within  the  various  job  families. 
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RESULTS 

Separate  analyses  were  conducted  on  the  predictions  based  on  the  job 
dimension  scores  for  the  individual  RAQs  and  for  each  set  of  clusters  as 
well  as  on  the  attribute  data.  In  each  case,  Pearson  product-moment 
correlations  were  computed  between  the  predicted  and  the  actual  test- 
related  measures  for  the  four  criteria. 

The  analyses  based  on  the  job  dimension  scores  were  originally 
conducted  for  the  total  sample  of  202  jobs.  As  had  been  the  case  in  past 
research,  predictions  relative  to  the  mean  test  scores  and  the  scores 
1 SO  below  the  mean  were  quite  good,  but  the  predictions  relative  to  the 
validity  coefficients  and  the  "val id-nonval id"  criterion  were  somewhat 
disappointing.  The  results  for  the  total  sample  will  not  be  reported 
here,  however,  because  of  certain  problems  that  were  found  to  be  assoc- 
iated with  certain  of  these  data.  (The  results  for  ihe  total  sample  are 
reported  by  McCormick,  DeNisi,  and  Shaw,  May  1977.) 

The  first  problem  steinned  from  the  fact  that  a large  portion  of  the 
sample  (79  jobs)  came  from  one  company,  hereafter  designated  as  "Company  X." 
The  test  data  from  this  company  were  all  based  on  "special"  or  in-house 
tests  developed  by  the  company,  which  it  was  feared  might  be  qualitatively 
different  from  the  other  tests  used  to  measure  the  different  constructs. 
Furthermore  the  test  data  from  this  company  from  the  79  jobs  were  consoli- 
dated into  seven  job  families,  and  it  was  not  possible  to  sort  out  those 
for  the  79  individual  jobs.  The  actual  test  data,  then,  were  available 
for  only  these  seven  job  families,  but  the  PAQ-based  predictions  were 
made  for  the  79  individual  jobs.  Each  individual  job  had  to  be  classified 
into  its  appropriate  job  family  and  the  test  data  for  the  job  family  were 
then  used  as  the  criterion  data  for  every  job  in  that  family.  It  was  felt 
that  this  procedure  may  have  allowed  too  much  "slippage"  and  might  tend 
to  reduce  any  correlation  between  predicted  and  obtained  criterion  data 
by  restricting  the  range  on  the  obtained  test  data.  It  was  therefore 
decided  that  eliminating  the  data  from  Company  X would  result  in  a "cleaner" 
analysis,  thus  providing  a truer  picture  of  the  predictive  ability  of  the 
PAQ  data. 

The  other  problem,  however,  was  not  solved  by  this  reduction  of  the 
sample.  The  problem  was  simply  that  there  were  a number  of  jobs  for  which 
actual  data  were  available  on  mean  test  scores,  but  not  on  scores  1 SD 
below  the  mean.  Therefore,  although  these  wo  criterion  indices  are  closely 
linked,  the  initial  analyses  for  these  two  criteria  were  conducted  on 
samples  that  overlapped  each  other,  but  th.jt  were  not  identical.  Thus 
some  differences  between  the  results  from  these  two  criteria  might  be  due 
to  the  differences  in  the  samples.  To  eliminate  this  second  problem  it 
was  decided  to  further  reduce  the  sample  by  including  in  the  analyses 
pertaining  to  the  mean  test  scores  and  the  scores  1 SD  below  the  mean,  only 
those  jobs  for  which  data  were  available  on  both  criteria.  The  results  that 
will  be  reported,  therefore,  are  those  from  the  analyses  conducted  on  this 
reduced  and  matched  sample.  This  sample  included  93  jobs.  These  results  are 
presented  in  Table  1. 
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Table  1 

Correlations  Between  Predicted  and  Actual 
Test-Related^ Criteria  for  Five  Constructs: 
Reduced  and  Matched  Sample 


Criterion  and 
Construct 

Individual 

PAOs 

Cluster-based  Predictions 
20  Clusters  40  Clusters  60  Clusters 

N 

Mean  Test  Scores 

General  Intelligence 

.7  A*** 

.52** 

.43** 

.52** 

33 

Verbal  Aptitude 

.71^^* 

.65*** 

.67*** 

.60*** 

50 

Numerical  Aptitude 

.67**^ 

. 52*** 

.61*** 

. 56*** 

64 

Spatial  Aptitude 

.74^** 

.44* 

.50** 

.47* 

26 

Clerical  Perception 

.53^ 

.44 

.37 

.37 

15 

Average 

.66 

.52 

.53 

.51 

1 SD  Below  Mean 

General  Intelligence 

.66*** 

.63*** 

.49** 

.53*** 

33 

Verbal  Aptitude 

.71*** 

.62*** 

,68*** 

.60*** 

50 

Numerical  Aptitude 

.63*** 

.47*** 

.59*** 

.55*** 

64 

Spatial  Aptitude 

. 76*** 

.47* 

.51** 

.50** 

26 

Clerical  Perception 

.60** 

.42 

.44 

.39 

15 

Average 

.68 

.53 

.55 

.52 

Validity  Coefficients 

General  Intelligence 

-.54 

.37 

-.14 

-.14 

13 

Verbal  Aptitude 

.30 

.53*** 

.58*** 

.28 

36 

Numerical  Aptitude 

.25* 

.12 

.48*** 

.40*** 

76 

Spatial  Aptitude 

.26 

.29 

.25 

.35* 

43 

Clerical  Perception 

-.02 

-.03 

.16 

.32* 

29 

Valid-Nonvalid 

General  Intelligence 

.17 

1 

— 

— 

13 

Verbal  Aptitude 

-.18 

— 

— 

— 

36 

Numerical  Aptitude 

.19 

— 

— 

— 

76 

Spatial  Aptitude 

•76*** 

— 

— 

— 

43 

Clerical  Perception 

.51** 

29 

^Analyses  for  cluster  based  predictions  of  valid-nonvalid  criterion  were 
not  carried  out. 


*Signi ficant,  p<  .05 

♦♦Significant,  p<.01 

♦♦♦Significant,  pc.OOl 


10 


As  can  be  seen  in  Table  1,  the  predictions  of  mean  test  scores  and 
scores  1 SD  below  the  mean  were  rather  successful  for  the  data  based  on 
both  individual  PAQs  and  the  data  based  on  the  clusters.  Looking  first 
at  the  prediction  of  mean  test  scores,  we  see  that  all  five  correlations 
for  the  individual  PAQs  are  significant  (four  of  them  at  the  .001  level) 
and  that  they  range  from  .53  (Clerical  Perception)  to  .74  (General 
Intelligence  and  Spatial  Aptitude)  the  average  correlation  for  the  five 
constructs  being  .66.  The  results  for  the  cluster-based  predictions  are  also 
quite  respectable.  However,  these  correlations  are  generally  a bit  lower 
than  for  the  individual  PAQs,  and  in  the  case  of  all  three  sets  of  cluster- 
based  predictions  the  correlations  for  Clerical  Perception  failed  to  reach 
significance.  Nevertheless,  the  average  correlations  for  the  predictions 
based  on  20,  40  and  60  clusters  are  .52,  .53,  and  .51  respectively; 
all  are  indicative  of  fairly  strong  relationships  between  predicted  and 
obtained  data. 

Looking  at  the  scores  1 SD  below  the  mean,  we  find  the  same  general 
pattern  of  results.  Again,  all  five  correlations  for  the  individual  PAQs 
are  significant  (four  at  the  .001  level),  the  range  being  from  .60 
(Clerical  Perception)  to  .76  (Spatial  Aptitude),  with  the  average  corr- 
elation being  .68.  Again,  the  cluster-based  predictions  are  also  fairly 
strong,  although  the  correlations  are  somewhat  lower  here  than  for  the 
individual  PAQs  and,  once  again,  none  of  the  correlations  for  Clerical 
Perception  is  significant.  The  average  correlations  for  the  predictions 
based  on  the  20,  40  and  69  clusters  are  .53,  .55  and  .52  respectively; 
slightly  better  than  for  the  mean  test  scores 

As  can  be  seen  in  Table  1,  however,  the  results  for  the  validity 
coefficients  and  the  val id-nonval id  criterion  are  rather  disappointing, 
especially  for  the  individual  PAQs.  The  correlations  for  the  individual 
PAQs  predicting  validity  coefficients  range  from  -.54  (General  Intelligence) 
to  .30  (Verbal  Aptitude)  with  only  one  significant  correlation.  Looking  at 
the  results  for  the  val id-nonval id  criterion,  we  see  they  are  a bit  better, 
the  range  being  from  -.18  (Verbal  Aptitude)  to  .76  (Spatial  Aptitude),  with 
two  of  the  correlations  being  significant  for  the  individual  PAQs.  Although 
no  analyses  were  conducted  with  the  val id-nonval id  criterion  for  the  cluster 
based  predictions,  it  is  interesting  to  look  at  the  cluster  results  for  the 
validity  coefficients.  The  results  for  the  20,  40  and  60  clusters  are 
generally  better  than  for  the  individual  PAQs.  One  would  expect  the  results 
based  on  the  individual  PAQs  to  be  superior  to  those  based  on  the  clusters 
since  the  cluster-based  analyses  provide  for  the  predictions  for  all  of  the 
individual  jobs  within  a given  cluster  to  be  the  same,  whereas  the  actual 
criterion  data  are  different  for  the  individual  jobs.  However,  since  PAQ- 
based  data  have  never  been  shown  to  be  very  successful  in  predicting  validity 
data,  it  may  be  that  much  of  the  variance  in  prediction  is  due  to  error 
variance,  and  that  by  averaging  these  predictions  for  a whole  cluster  one  is 
simply  eliminating  some  of  that  error  variance. 

Before  moving  to  the  predictions  based  on  the  attribute  data,  one 
further  point  should  be  made.  Looking  at  the  results  in  Table  1,  we 
notice  that  the  results  obtained  for  the  three  different  sets  of  cluster- 
based  predictions  are  substantially  the  same.  This  is  interesting 


because  one  would  expect  that  by  increasing  the  number  of  clusters  to  60, 
the  resulting  cluster  predictions  would  more  nearly  approximate  the  pre- 
dictions for  the  individual  PAQs.  Conversely,  one  might  expect  that  the 
predictions  based  on  20  clusters  (in  which  the  jobs  in  each  cluster  are 
more  heterogeneous  than  in  the  case  of  40  or  60  clusters)  would  be  some- 
what lower  because  of  greater  possible  variability  in  the  jobs  within 
the  individual  clusters.  It  is  true  that  the  individual  PAQs  generally 
do  better  than  the  cluster-based  predictions,  but  when  considering  only 
the  cluster  data,  we  see  that  the  three  sets  of  cluster-based  predictions 
do  not  differ  substantially  from  one  another. 

In  the  past  there  has  been  a great  deal  of  concern  with  how  one  can 
determine  the  optimal  number  of  clusters  to  use  from  an  iterative  grouping 
procedure  such  as  the  one  used  here.  These  results  indicate,  however, 
that  at  least  for  use  in  a job  component  validity  model,  this  may  not  be 
a crucial  consideration.  Further  research  fs  needed,  of  couse,  but  these 
findings  indicate  that  a researcher  may  be  able  to  rely  more  on  practical 
considerations  in  choosing  the  optimal  cluster  solution  for  use  in  a job 
component  validity  model. 

Finally,  turning  to  the  predictions  based  on  the  attribute  data. 

Table  2 presents  the  results  of  the  predictions  for  selected  attributes  of 
the  different  constructs  for  the  criteria  of  mean  test  scores  and  scores 
1 SO  below  the  mean.  The  results  presented  here  are  based  on  the  complete 
sample  of  202,  since  the  inconsistency  of  the  results  did  not  seem  to 
warrant  further  analyses  with  a reduced  and/or  "matched"  sample.  As  can 
be  seen  in  Table  2,  althought  the  predictions  based  on  a few  of  the  attri- 
butes are  quite  respectable  (especially  in  predicting  mean  test  scores  for 
Clerical  Perception),  others  are  quite  poor  (especially  in  predicting 
either  criterion  index  for  Spatial  Aptitude).  The  inconsistencies  in  the 
results  case  some  doubt  on  the  utility  of  attribute  data  in  a job  compon- 
ent validity  model.  However,  the  fact  that  some  attribute  predictions 
were  quite  good  might  suggest  that  further  research  could  be  useful  in 
identifying  the  particular  circumstances  in  which  attribute  data  might  be 
a useful  basis  for  predicting  test  requirements. 
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Table  2 

Correlations  Between  Selected 
Attribute  Data  and  Criterion  Data 
for  the  Total  Sample 


Criterion  and  Construct 

Attribute 

Correlation 

N 

Mean  Test  Scores 

General  Intelligence 

Intelligence 

-.07 

111 

Verbal  Aptitude 

Verbal  Comprehension 

• 42^^ 

50 

Verbal  Aptitude 

Work  Fluency 

50 

Verbal  Aptitude 

Oral  Communication 

.43^^ 

50 

Numerical  Aptitude 

Numerical  Computation 

. 33^^^ 

163 

Numerical  Aptitude 

Arithmetic  Reasoning 

-.31^^^ 

163 

Spatial  Aptitude 

Visual  Perception 

-.31^^^ 

125 

Spatial  Aptitude 

Spatial  Visualization 

.,22*** 

125 

Spatial  Aptitude 

Spatial  Orientation 

.32*** 

125 

Clerical  Perception 

Verbal  Comprehension 

.63*** 

38 

Clerical  Perception 

Arithmetic  Reasoning 

.62*** 

38 

Clerical  Perception 

Closure 

. 66*** 

38 

Clerical  Perception 

Visual  Perception 

.^2** 

38 

1 SD  Below  Mean 

General  Intelligence 

Intelligence 

.01 

no 

Verbal  Aptitude 

Verbal  Comprehension 

.43^^ 

50 

Verbal  Aptitude 

Work  Fluency 

50 

Verbal  Aptitude 

Oral  Communication 

.44^^ 

50 

Numerical  Aptitude 

Numerical  Computation 

.26^^ 

141 

Numerical  Aptitude 

Arithmetic  Reasoning 

.23^^ 

141 

Spatial  Aptitude 

Visual  Perception 

-.27** 

103 

Spatial  Aptitude 

Spatial  Visualization 

-.32*** 

103 

Spatial  Aptitude 

Spatial  Orientation 

-.28** 

103 

Clerical  Perception 

Verbal  Comprehension 

.51 

15 

Clerical  Perception 

Arithmetic  Reasoning 

.36 

15 

Clerical  Perception 

Closure 

.16 

T5 

Clerical  Perception 

Visual  Perception 

-.06 

15 

♦Significant,  p<.05 
♦♦Significant,  p<.01 
♦♦♦Significant,  p<.001 
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CONCLUSIONS 


On  the  basis  of  the  results  of  this  study  the  following  conclusions 
seem  to  be  warranted  regarding  the  use  of  data  from  the  Position  Analysis 
Questionnaire  (PAQ)  in  the  job  component  validity  model  as  the  basis  for 
establishing  aptitude  requirements  for  use  in  personnel  selection. 

1.  Such  a model  can  serve  to  identify  the  aptitude  tests  that  have 
substantial  validity  for  use  in  personnel  selection.  This  is  done  on 
the  basis  of  statistical  analyses  of  data  from  PAQ  analyses  of  individual 
jobs.  This  conclusion  is  supported  particularly  by  the  findings  regarding 
the  predictability  of  mean  test  scores  of  job  incumbents  on  various  jobs, 
and  the  scores  of  job  incumbents  one  standard  deviation  below  the  mean. 

This  is  especially  true  when  using  a reduced  and  "matched"  sample  which 
was  probably  the  most  representative  sample  available.  Such  predictions, 
originally  based  on  test  data  for  incumbents  on  the  nine  tests  of  the 
General  Aotitude  Test  Battery  (GATB)  of  the  United  States  Employment 
Service,  also  hold  up  quite  well  with  test  data  from  various  commercial 
tests  that  were  considered  to  measure  the  same  constructs  as  those  measured 
by  the  GATB  tests.  (This  analysis  was  based  on  five  of  the  nine  constructs.) 
Results  of  the  predictions  based  on  commercial  tests  resulting  from  this 
study  are  further  supported  by  a recent  study  by  Cunningham  et  al.  (1976) 

in  which  the  Differential  Aptitude  Tests  were  used  in  much  the  same  fashion. 

2.  The  predictions  of  the  validity-related  criteria  (those  consisting 
of  validity  coefficients  and  those  based  on  a "valid-nonvalid"  determination) 
were  generally  not  very  satisfactory.  These  results  are  generally  consis- 
tent with  certain  previous  studies  in  which  the  prediction  of  validity 
coefficients  also  was  rather  poor. 

3.  Predictions  of  mean  test  scores  and  scores  one  standard  deviation 
below  the  mean  that  are  based  on  job  families  (formed  from  PAQ  data)  are  a 
bit  lower  than  those  based  on  PAQs  for  individual  jobs.  They  are,  however, 

of  such  magnitude  as  to  warrant  further  possible  research  in  the  job  component 
validity  model.  The  predictions  of  the  criterion  of  validity  coefficients 
based  on  job  family  data  actually  tended  to  be  slightly  better  than  the 
predictions  based  on  PAQ  analyses  of  individual  jobs. 

4.  Predictions  based  on  job  families  resulting  from  the  20,  40,  and 
60  cluster  solutions  were  virtually  identical  to  each  other.  This  is 
interesting  since  one  of  the  problems  that  has  often  been  discussed  re- 
lative to  the  use  of  hierarchical  grouping  procedures  has  been  the  decision 
regarding  the  "optimal"  cluster  solution.  The  results  from  this  study 
suggest  that  the  number  of  job  families  used  in  the  job  component  validity 
model  may  not  be  critical,  although  further  research  is  clearly  needed  in 
this  area. 

5.  In  connection  with  the  criteria  of  mean  test  scores  and  the  scores 
one  standard  deviation  below  the  mean,  predictions  of  the  test-related 
criteria  from  the  attribute  data  are  not  nearly  as  consistent  as  those  from 
the  job  dimension  scores  based  on  PAQ  analyses.  Although  the  predictions  from 
the  attribute  data  were  reasonably  good  for  certain  constructs,  they  were 
very  poor  in  the  case  of  others.  Such  inconsistencies  have  been  found  in 
previous  studies  as  well.  It  would  seem,  therefore,  that  future  research 
relating  to  the  use  of  attribute  data  for  predicting  aptitude  requirements  of 
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jobs  might  well  be  focused  on  the  identification  of  the  particular  attributes 
for  which  such  predictions  can  be  made  with  reasonable  validity. 

6.  In  summary,  although  previous  research  with  the  use  of  the  PAQ  as 
the  basis  for  establishing  aptitude  requirements  for  jobs  within  the  job 
component  validity  framework  has  dealt  exclusively  with  test  data  from  the 
6ATB  tests,  the  results  of  this  study  indicate  quite  clearly  that  such  data 
can  also  be  used  in  the  establishment  of  aptitude  requirements  in  terms  of 
commercial  tests  that  presumably  measure  the  same  constructs. 
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